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2  Introduction 


The  results  reported  herein  represent  a  significant  step  in  developing  a  decision 
model  and  software  tool  to  support  affordable  Naval  acquisition  planning  in  the 
presence  of  uncertainty  and  severely  constrained  budgets.  Navy  decision  makers 
are  faced  with  difficult  tradeoffs  among  different  technology  investments,  each 
promoted  by  their  own  subject  matter  experts.  In  order  to  arrive  at  the  right  trade¬ 
offs  it  is  essential  to  place  each  proposed  technology  investment  in  the  overall 
context  of  successfully  supporting  future  Navy  missions.  We  aimed  to  develop  a 
tool  to  do  just  that.  In  addition,  we  aimed  to  help  decision  makers  gain  insight  into 
what  is  fundamentally  a  complex  problem  by  supporting  ‘what-if’  type  questions 
regarding  the  decisions  at  hand,  such  as:  “Which  technologies  do  I  need  to  fund  in 
order  to  satisfy  a  specific  Navy  objective  with  high  probability?”  or  “How  should 
I  structure  my  investments  in  order  to  guard  against  the  possibility  of  a  technol¬ 
ogy  failure  in  a  certain  high-risk/high-payoff  investment?”  and,  most  importantly, 
“What  is  a  good  balanced  investment  strategy  to  address  the  full  range  of  expected 
future  Navy  needs?”. 

To  help  answer  such  questions,  we  developed  a  software  prototype  (called 
AADA-Affordable  Acquisition  Decision  Aid)  which 

•  supports  investment  decisions  among  highly  diverse  proposed  technology 
investments 

•  measures  future  asset  performance  with  respect  to  a  diverse  range  of  high- 
level  Navy  objectives 

•  accounts  for  technological  risks,  uncertainty  in  future  objectives  and  threats, 
and  uncertainty  in  future  budgets 

•  employs  genetic  algorithms  to  determine  the  utility  of  future  assets  by  per¬ 
forming  near-optimal  allocation  of  assets  to  objectives 

AADA  is  intended  to  integrate  the  outputs  of  performance  models,  cost  mod¬ 
els,  and  other  simulation-based  acquisition  tools  to  improve  the  quality  of  acqui¬ 
sition  decisions,  aiming  to  strengthen  Navy  warfighting  capability  while  reducing 
total  ownership  cost. 

We  met  the  following  specific  objectives: 

•  We  extended  the  underlying  mathematical  model  with  respect  to  the  model 
resulting  from  Phase  I  (see  section  3). 
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•  We  improved  the  scope  and  performance  of  the  optimization  algorithms  (see 
section  4). 

•  We  produced  a  detailed  system  design  plan  for  a  network  centric  acquisition 
decision  aid  and  a  software  prototype  to  meet  this  design  plan  (see  section 
5). 

•  We  developed  several  Navy-specific  acquisition  examples  to  test  the  soft¬ 
ware  prototype  and  to  aid  in  transitioning  the  research  towards  a  marketable 
product  (see  sections  6  and  6.3). 

In  designing  and  completing  the  network  centric  software  prototype,  we  suc¬ 
ceeded  in  delivering  a  product  that  had  been  originally  scheduled  for  delivery  in 
a  Phase  II  Option.  This  software  prototype  has  proved  valuable  in  our  efforts  to 
demonstrate  the  benefits  of  our  approach  to  potential  customers. 


3  Extensions  to  the  Mathematical  Model 

In  Phase  I  we  had  successfully  demonstrated  how  a  mathematical  model  describ¬ 
ing  the  optimal  allocation  of  a  fixed  collection  of  assets  to  targets  could  be  gener¬ 
alized  to  a  model  for  optimal  investment  decisions. 

In  order  to  more  faithfully  model  realistic  acquisition  decision  processes,  we 
needed  to  broaden  the  scope  of  the  mathematical  model  to  encompass  the  follow¬ 
ing  important  concepts: 

•  We  generalized  our  concept  of  a  ‘target’  to  the  more  general  concept  of 
‘objective’. 

•  We  introduced  the  concept  of  scenarios  to  support  acquisition  planning  for 
a  flexible  force  structure  that  can  address  diverse  requirements  and,  in  par¬ 
ticular,  to  better  model  the  fact  that  in  multiple  possible  future  worlds  faced 
by  the  US  Navy,  different  objectives  take  on  different  importances. 

•  We  introduced  a  capability  to  model  varying  degrees  of  technological  risk 
associated  with  different  investments. 

•  We  introduced  a  measure  for  the  expected  impact  on  Total  Ownership  Cost 
of  each  investment  when  successfully  deployed. 

In  the  following,  the  resulting  mathematical  model  is  elaborated  in  detail. 
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Figure  1 :  Investment  Evaluation 


3.1  Detailed  Mathematical  Model 

Given  an  available  budget  B  and  a  choice  of  n  S&T  investments  I  —  {ti,  t2,  ■  ■  ■ ,  tn}, 
each  with  a  proposed  budget  b{,  choose  an  S&T  strategy,  defined  as  a  subset 
SCI,  such  that  the  expected  S&T  cost/performance  cps(S)  is  maximal  under 
the  constraint 

£  <  B. 

ues 

S&T  cost  performance  (detailed  in  the  following)  accounts  for  uncertainty 
of  S&T  results  and  the  impact  of  successful  S&T  on  asset  properties  such  as  pre¬ 
dicted  effectiveness  in  achieving  multiple  objectives,  predicted  risk  while  engaged 
in  achieving  those  objectives,  and  predicted  TOC. 

Given  a  fixed  S&T  strategy  S  =  {ti,t2,  ■ . .  ,tm},  and  a  probability  pt  of  suc¬ 
cess  for  each  such  investment,  the  potential  results  of  that  strategy  are  all  the 
subsets  R  C  S,  each  with  probability 


p(R)  = 


na- 

ti£R 


> 


so  that  the  cost/performance  of  strategy  S  is  expressed  as 


cps{S )  =  p(R)cPr(R ), 

RCS 
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Figure  2:  Asset  Evaluation 


where  cpr(R)  is  the  predicted  cost/performance  of  assets  after  successfully  com¬ 
pleting  exactly  those  S&T  programs  contained  in  R. 

Given  an  S&T  result  R  =  {fi,  t2, . . . ,  tj},  i.e.,  a  set  of  successful  S&T  pro¬ 
grams,  we  expect  to  transition  the  results  of  those  programs  in  order  to  improve 
the  cost/performance  of  existing  assets.  In  some  circumstances,  however,  we  may 
choose  to  fund  multiple  complementary  high-risk  S&T  programs  merely  to  max¬ 
imize  the  likelihood  for  at  least  one  of  them  to  succeed.  Should  more  than  one 
succeed,  it  may  not  be  advantageous  to  transition  all  of  the  successes.  We  there¬ 
fore  define  the  cost/performance  of  S&T  results  as  follows 

cPr(R)  =  maxcpr(T) 

1  L.rL 

where  cpr{T )  is  the  predicted  cost/performance  of  assets  after  transitioning  ex¬ 
actly  those  successful  S&T  programs  contained  in  T.  Figure  1  depicts  the  com¬ 
plete  subset  structure  of  possible  investments  and  potential  outcomes  to  be  evalu¬ 
ated. 

When  evaluating  a  collection  T  =  {fi,  i2,  •  •  • ,  h)  of  successfully  transitioned 
S&T  programs,  we  measure  the  cost/performance  of  T  in  terms  of  its  impact  on 
a  collection  of  existing  assets  A  =  {ai,  a2, . . . ,  a*}.  In  particular,  the  following 
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parameters  are  considered: 

effridi,  Oj)  effectiveness  of  asset  a*  vs.  objective  Oj 
rskT(a,i,  Oj)  risk  of  deploying  asset  at  vs.  objective  Oj 
tocr(ai)  annualized  Total  Ownership  Cost  of  asset  a. 

We  define  the  resulting  TOC  change  A  toe  as 

A  toe  =  1  -  (13  tocriai ))  /  (13  toc<D(ai))  > 

i.e.,  as  a  relative  improvement  compared  to  the  status  quo. 

Now  the  combined  cost/performance  of  T  is  defined  as  the  weighted  sum  of 
cost  improvement  and  predicted  asset  performance 

w toc  A  toe  +  WperfperfT(A,  H) 

for  a  suitable  choice  of  weights  wtoc  and  Wperf  and  a  suitable  performance  mea¬ 
sure  perfT(A,  H )  of  upgraded  assets  versus  a  hierarchy  H  of  anticipated  objec¬ 
tives. 

Asset  performance  is  measured  with  respect  to  a  mix  of  scenarios,  defined  in 
a  hierarchy  (see  figure  2).  At  each  level  H  of  the  scenario  hierarchy  with  subordi¬ 
nate  scenarios  Hi,  H2, . . . ,  Hn,  the  overall  scenario  performance  is  expressed  as  a 
weighted  sum  of  individual  scenario  performances, 


perfT(A,  H)  =  13  WipHiperfT{A,  Hi) 

for  a  suitable  choice  of  relative  scenario  importances  Wi  and  conditional  probabil¬ 
ities  pHi  of  encountering  scenario  Hi  within  scenario  H. 

A  basic  scenario  K  is  defined  in  terms  of  the  relative  importances  it  assigns  to 
individual  objectives  oi,  02, . . . ,  o*.  Denote  scenario-dependent  objective  impor¬ 
tance  as  impK(oj).  Denote  asset  importance  for  assets  01, 02, . . . ,  a*  as  imp(ai). 
Asset  performance  with  respect  to  such  scenarios  is  determined  by  allocating  indi¬ 
vidual  assets  to  objectives  so  as  to  maximize  the  effectiveness  in  achieving  objec¬ 
tives  while  minimizing  risk.  An  asset  allocation  is  denoted  as  a  mapping  /  where 
f(i)  =  j  when  asset  i  is  allocated  to  objective  j.  Multiple  assets  may  be  allocated 
to  one  objective  to  increase  the  likelihood  of  achieving  that  objective. 

We  now  define  the  risk  component  for  an  asset  allocation  /  with  respect  to 
scenario  K : 

rskKj  =  YjrskT(ai,of(i))imp(ai). 
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The  effectiveness  component  must  account  for  diminishing  returns  when  allocat¬ 
ing  multiple  assets  to  the  same  objective: 

effKj  =  E  ( 1  “  II  (!  “  effr(ai,  Oj))  J  impK(oj). 
j  \  m=j  J 

Now  we  can  express  asset  performance  with  respect  to  a  basic  scenario  as 
perfT(A,  K)  =  max  (weeffxj  —  wrrskx,f) 

for  suitable  weights  we  and  wT  for  effectiveness  and  risk,  respectively.  This  con¬ 
cludes  the  elaboration  of  performance  evaluation. 

4  Optimization  Methods 

4.1  Background 

A  special  strength  of  our  work  has  been  the  ability  to  make  use  of  MAAPtm,  a 
Prometheus  technology  for  optimizing  the  allocation  of  assets  to  threats  based 
on  expert  knowledge  of  operational  parameters  of  assets  and  characteristics  of 
threats.  MAAP  is  an  automated  decision  aid  developed  for  the  Air  Force  to  help 
automate  the  real-time  allocation  of  aircraft  to  targets  (hence  the  acronym  MAAP 
-  Military  Aircraft  Allocation  Planner)  in  an  optimal  or  near-optimal  manner. 
MAAP  is  readily  customizable  for  a  broad  class  of  allocation  problems  and  has 
been  demonstrated  to  operate  effectively  on  large  problems  (400  assets  by  400 
targets).  MAAP  combines  EDM,m,  the  Extended  Dependency  Model  originally 
developed  by  the  Principal  Investigator  to  measure  the  effectiveness  of  the  Trident 
Command  and  Control  System  [3],  with  a  genetic  algorithm  (GA)-based  opti¬ 
mization  engine.  This  optimization  engine  continues  to  represent  the  core  of  the 
AADA  optimizer. 

During  Phase  II  of  this  project,  we  further  developed  this  technology  to  ac¬ 
commodate  the  extended  mathematical  model  described  in  section  3.  The  AADA 
optimizer  has  been  structured  in  a  hierarchy  of  layers  which  successively  break 
up  the  overall  optimization  problem  into  simpler  problems.  The  innermost  layer, 
the  Scenario  Level  Optimizer,  draws  on  the  MAAP  allocation  engine. 


4.2  Structure  of  the  Optimizer 

The  complete  list  of  layers  is  given  in  the  following  (top-to-bottom): 
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1.  S&T  Level  Optimizer 

(a)  S&T  funding  layer 

Enumerates  the  maximal  affordable  collections  of  S&T  programs,  i.e., 
the  sum  of  their  budgets  meet  the  budget  constraints  and  no  further 
programs  can  be  added  without  violating  the  budget  limit. 

(b)  S&T  uncertainty  layer 

For  each  such  collection  of  fundable  programs,  accounts  for  the  un¬ 
certainty  in  S&T  by  considering  all  subsets  of  successful  programs  in 
accordance  with  the  combined  probability  of  the  subset. 

(c)  Upgrade  deployment  layer 

For  each  collection  of  successful  S&T  programs,  considers  all  subsets 
of  resulting  upgrades  which  may  now  be  considered  for  deployment. 

(d)  Objective  performance  layer 

For  each  subset  of  deployed  upgrades,  determines  its  impact  on  TOC 
and  balances  cost  against  the  resulting  performance  of  upgraded  assets 
versus  existing  threats.  Asset  performance  is  measured  with  respect  to 
each  of  the  scenarios  in  the  objective  hierarchy  and  results  are  com¬ 
bined  in  a  weighted  sum  depending  upon  user  specified  importance 
values  and  probabilities. 

2.  Scenario  Level  Optimizer 

(a)  Asset  allocation  layer 

Employs  previously  developed  Prometheus  GA  algorithms  to  rapidly 
determine  near-optimal  allocations  of  collections  of  assets  to  objec¬ 
tives. 

(b)  Objective  function 

For  a  given  allocation  of  assets  to  objectives,  determine  its  overall 
value. 

4.3  Performance  considerations 

The  two  most  important  performance  concerns  in  the  present  prototype  are  the 
performance  of  the  innermost  loops  and  the  combinatorial  aspects  of  the  outer  op¬ 
timization  layer.  As  for  the  former,  we  were  able  to  adapt  proprietary  Prometheus 
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algorithms  which  had  already  been  heavily  optimized.  To  limit  the  combinato¬ 
rial  explosion  in  the  outer  layer,  we  employed  dynamic  programming  techniques 
which  ensure  that  any  overlap  which  could  result  from  evaluating  subsets  (de¬ 
ployments)  of  subsets  (S&T  successes)  of  subsets  (funded  S&T  programs)  of  a 
collection  of  S&T  programs  considered  for  funding  is  avoided. 

As  a  result,  the  prototype  produces  a  recommended  S&T  funding  strategy  for 
our  USW  example  (see  section  6.3)  in  approximately  30  minutes  on  a  900MHz 
AMD  Athlon  PC  running  Linux.  This  example  covers  7  scenarios,  10  different 
objectives,  8  existing  assets  and  13  potential  technology  investments. 

The  most  important  remaining  performance  limitation  is  that  the  number  of 
individual  investments  may  not  get  very  large.  The  limit  when  running  on  work¬ 
station  class  hardware  is  about  20  different  candidate  investments.  We  expect  to 
push  this  boundary  beyond  100  investments  in  Phase  III. 


5  System  Design  and  Prototype  Software 

A  principal  accomplishment  of  this  Phase  II  project  has  been  the  completion  of 
a  runnable  Web-based  prototype  client/server  software  system  for  specifying  and 
running  S&T  type  optimization  problems.  The  client  component  is  completely 
written  in  Java  and  can  be  run  on  a  large  range  of  platforms,  from  workstations  and 
desktops  to  handheld  devices  and  emerging  Internet  appliances.  The  server  soft¬ 
ware  consists  mostly  of  a  C-based  optimization  package  and  is  presently  hosted 
on  a  Linux-based  server. 

5.1  Client 

The  AADA  client  software  performs  two  major  functions,  as  described  in  the 
System  Design  Plan  (see  appendix  A.)  It  serves  as  an  acquisition  scenario  editor 
to  specify  complex  aquisition  problems  and  it  provides  a  postoptimization  analysis 
capability  to  inspect  and  evaluate  system  recommendations. 

For  the  first  of  these  functions,  the  System  Design  Plan  called  for  three  layers 
responsible,  respectively,  for  defining  objectives,  existing  assets,  and  S&T  pro¬ 
grams  for  developing  asset  upgrades.  In  developing  the  prototype  software,  we 
have  found  it  useful  to  further  subdivide  the  first  of  these  layers  and  to  provide 
separate  layers  for  specifying  basic  objectives  on  one  hand  and  the  scenarios  as¬ 
signing  importances  to  those  basic  objectives  (depending  on  a  user-defined  con¬ 
text)  on  the  other. 
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Figure  3:  Objective  layer  (top)  and  scenario  layer  (bottom) 

In  the  following,  we  will  illustrate  the  various  layers  with  screenshots  from 
the  initial  version  of  our  ASW  in  the  Littoral  example  (see  section  6.1). 

Layer  I:  Objectives.  The  objective  layer  (figure  3)  permits  editing  of  a  list  of 
the  future  objectives  which  must  be  addressed  by  future  assets.  Performance  of 
potential  asset  upgrades  is  measured  with  respect  to  predicted  improvements  in 
meeting  those  objectives.  An  objective  may  be,  for  example,  to  counter  a  specific 
enemy  threat,  or  to  perform  a  specific  task. 

Layer  II:  Scenarios.  Depending  on  a  given  situation,  individual  basic  objectives 
may  become  more  or  less  important.  Future  assets  must  be  sufficiently  flexible  to 
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perform  in  a  range  of  diverse  situations.  These  might  include,  for  example,  spe¬ 
cific  regional  conflicts,  particular  phases  in  a  Naval  campaign,  or  the  emergence 
of  a  possible  but  uncertain  threat.  Scenarios  may  be  nested  to  any  depth  so  that 
we  could,  for  instance,  specify  the  phases  of  a  campaign  responding  to  a  potential 
threat  emerging  during  a  specific  regional  conflict.  For  each  scenario,  its  likeli¬ 
hood  and  its  relative  importance,  if  it  occurs,  must  be  specified.  All  probabilities 
are  conditional  with  respect  to  the  occurrence  of  the  parent  scenario. 

Layer  III:  Assets.  Here  the  user  may  specify  the  characteristics  of  existing  assets 
(see  figure  4).  For  each  type  of  asset  listed,  the  following  parameters  must  be 
given: 

•  the  number  of  existing  assets  of  this  type 

•  the  value  of  each  asset  (measuring  the  negative  impact  of  losing  such  an 
asset  relative  to  other  asset  types) 

•  an  estimate  of  present  TOC  per  platform 

Of  special  importance  are  parameters  P*  specifying  the  degree  of  success  that  can 
be  expected  when  allocating  one  asset  of  the  given  type  to  each  of  the  basic  objec¬ 
tives.  During  optimization,  multiple  assets  may  be  allocated  to  the  same  objective 
to  increase  overall  performance.  Similarly,  parameters  PCk  specify  the  likelihood 
of  outright  loss  of  an  asset  when  allocated  to  each  of  the  basic  objectives.  and 
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Figure  5:  Upgrade  layer 

Pck  are  numbers  between  0  and  1,  with  0  representing  ‘no  impact’  and  1  repre¬ 
senting  ‘complete  success’  or  ‘certain  loss’,  respectively. 

Layer  IV:  Upgrades.  After  defining  the  framework  for  acquisition  decisions  in 
Layers  I-III,  the  range  of  potential  acquisition  choices  is  defined  in  the  upgrade 
layer  (figure  5).  Before  being  able  to  deploy  an  upgrade  to  existing  assets,  an  S&T 
investment  is  required.  For  each  upgrade  under  consideration,  the  user  specifies 
its  S&T  cost.  Total  S&T  costs  must  stay  strictly  within  a  fixed  budget.  S&T 
does  not  result  in  a  deployable  upgrade  with  certainty,  but  with  a  user-defined 
probability.  When  successful,  the  resulting  upgrade  is  characterized  by  its  effect 
on  asset  parameters,  in  particular,  the  predicted  impact  on  TOC,  Pk,  and  PCk- 

The  above  four  layers  together  comprise  the  acquisition  scenario  editor  and 
allow  the  definition  of  complex  decision  problems.  After  such  a  problem  has  been 
specified,  an  ‘Optimize’  function  may  be  activated  from  a  system  menu.  This 
transfers  a  machine  representation  of  the  optimization  problem  to  a  server  (see 
section  5.2).  The  server  performs  the  computations  required  to  determine  acquisi¬ 
tion  recommendations  which  are  transmitted  back  to  the  client  and  presented  for 
postoptimization  analysis,  described  in  the  following. 
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Figure  6:  Output  layer 


Layer  V:  Optimizer  Output.  While  the  previous  layers  dealt  with  data  entry, 
the  output  layer  (figure  6)  permits  reviewing  of  results.  The  system  presents  its 
top  three  acquisition  recommendations,  as  well  as  an  evaluation  of  a  hypothetical 
‘no-upgrade’  option,  to  help  visualize  projected  improvements  versus  the  status 
quo.  An  overview  screen  lists  total  S&T  cost  for  the  proposed  upgrades,  a  mea¬ 
sure  of  the  expected  overall  performance  versus  the  stated  objectives,  as  well  as 
expected  overall  TOC.  A  second  screen  details  the  upgrades  recommended  for 
S&T  funding,  along  with  their  S&T  cost. 

5.2  Server 

The  bulk  of  the  server  consists  of  C-based  optimization  software  structured  along 
the  lines  described  in  section  4.2  The  interface  between  the  optimization  code  and 
the  Java  client  is  provided  by  a  small  Java  package  using  the  Java  RMI  (remote 
method  invocation)  facility.  The  server  also  manages  access  to  the  AADA  system 
for  multiple  users.  In  the  short  term,  this  has  proven  useful  for  purposes  of  demon¬ 
strating  the  AADA  software  to  potential  customers.  In  the  long  term  we  expect  to 
support  multi-user  collaboration  on  complex  acquisition  decision  problems. 


13 


6  Navy-specific  acquisition  examples 

Three  examples  were  developed  in  the  course  of  this  program.  These  examples 
were  used  to  demonstrate  the  power  and  utility  of  AADA.  The  first  is  labeled 
ASW.  The  second,  an  example  of  programs  or  exploratory  developments  to  help 
the  Navy  deter  and  counter  weapons  of  mass  destruction,  is  labeled  WMD.  The 
third  example,  called  USW,  enhances  the  first  by  adding  new  levels  of  complexity. 

6.1  ASW 

This  first  example  was  used  to  develop  and  demonstrate  the  AADA  concept.  It 
is  representative  of  a  class  of  acquisition  decisions  that  program  managers  regu¬ 
larly  encounter.  Furthermore  this  type  of  problem  cannot  be  solved  using  linear 
programming.  A  genetic  algorithm  or  other  non-linear  technique  is  needed.  The 
structure  of  AADA  emerged  from  this  example,  which  constituted  our  first  proto¬ 
type. 

The  context  of  the  problem  includes  two  operational  theatres  with  six  threats, 
or  objectives.  The  asset  mix  includes  air,  surface  and  subsurface  ASW  platforms. 
There  are  six  potential  technology  upgrades,  two  for  each  asset  class.  The  problem 
is  to  decide  which  portfolio  of  technology  upgrades  within  a  fixed  budget  would 
provide  the  most  performance  while  minimizing  total  ownership  costs. 

A  generic  description  of  the  problem,  its  structure,  and  the  relationships  of  the 
various  factors  can  be  found  in  appendix  B,  “Affordable  Acquisition  Decision  Aid 
(AADA)  Descriptive  Guide  for  Analysts.” 

6.2  WMD 

This  problem  was  defined  to  demonstrate  the  ability  of  the  AADA  process  to  ad¬ 
dress  one  of  the  four  strategic  concepts  of  Submarine  Undersea  Warfare.  Counter 
and  Deter  Weapons  of  Mass  Destruction,  as  a  strategic  concept,  is  at  a  higher 
level  of  abstraction  than  the  objective  in  the  ASW  example.  This  example  di¬ 
rectly  addresses  concerns  faced  by  the  program  manager  for  submarine  undersea 
warfare  technologies  (NAVSEA93).  Therefore,  it  serves  as  a  demonstration  that 
AADA  and  its  accompanying  analytical  process  can  be  used  to  provide  structure 
to  a  problem  area  that  had  been  devoid  of  approaches. 

The  WMD  problem  includes  chemical,  biological  and  nuclear  delivery  vehicle 
threats;  near-shore,  inland,  and  shipbome.  Two  different  geographical  areas  are 
considered  as  well  as  a  combined  conflict.  The  assets  include  Unmanned  Aerial 
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Vehicles  (UAV)  configured  for  supporting  neutralization  of  ballistic  missiles,  un¬ 
supported  ground  surveillance,  and  signal  intelligence;  Special  Operations  Forces 
with  biological/chemical  and  nuclear  delivery  vehicle  neutralization  equipment; 
Tomahawk  Land  Attack  strike,  Anti-Surface  Ship  strike,  and  Anti-Ballistic  Mis¬ 
sile  (ABM)  missiles;  and  platform  organic  signal  intelligence. 

The  effort  here  showed  that  AADA  can  be  used  successfully  at  high  levels 
of  abstraction.  Details  of  this  effort  were  provided  in  presentations  to  ONR,  the 
Undersecretary  of  Defense  for  AT&L,  and  NAVSEA93  Analysts. 

6.3  USW 

The  foundational  ASW  problem  was  enhanced  with  a  third  operational  environ¬ 
ment  (the  Yellow  Sea),  a  new  warfare  area  (undersea  mine  warfare),  and  a  new 
class  of  asset  (a  joint  asset).  This  new  scenario  is  a  combined  ASW/Mine/Undersea 
warfare  problem  in  three  operational  locations.  The  asset  classes  are  air  (P3,  SH- 
60R),  submarine  (6881,  SEAWOLF,  VIRGINIA),  surface  (DD,  DDG,  FFG),  and 
coordinated  assets.  Coordinated  assets  are  combinations  of  two  or  more  of  the 
previously  mentioned  platform  types.  Of  thirteen  proposed  technology  upgrades, 
eight  are  solely  ASW  improvements,  three  are  solely  mine  improvements  and  two 
are  improvements  in  both  ASW  and  Mine  warfare.  Five  of  these  technology  up¬ 
grades  are  specific  to  one  type  of  platform,  five  are  specific  to  two  types  and  three 
are  applicable  to  all  asset  platforms. 

These  USW  enhancements  were  stimulated  by  participation  at  the  NDIA  Un¬ 
dersea  Warfare  Conference  in  September  2000,  during  which  CNO  N74  presented 
an  overview  of  a  current  Undersea  Warfare  problem.  In  this  conference  multiple 
presenters  reiterated  a  common  acquisition  problem: 

•  There  is  not  enough  money  to  do  everything. 

•  No  existing  methodology  supports  balanced  investment  decision-making 
and  ties  it  to  effects-based  mission  performance  in  the  field. 

Using  this  scenario,  we  demonstrate  that  AADA  can  provide  a  balanced  set 
of  investment  strategies  to  optimize  operational  effectiveness  across  an  extremely 
complex  decision  problem. 
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7  Transition  Efforts 


7.1  ONR 

Potential  for  AADA  application  to  ONR  programs  has  been  and  continues  to 
be  investigated  in  coordination  with  the  program  technical  advisor  at  ONR,  Ms. 
Katherine  Drew,  Code  362.  Two  formal  briefings  and  several  informal  collabo¬ 
rative  technical  discussions  have  failed  to  identify  a  specific  ONR  program  that 
has  a  decision  problem  that  correlates  well  to  the  AADA  multiple  objective  tree 
structure. 

7.2  NAVSEA93-SUBTECH 

RADM  C.  Young  and  his  functionary  staff  (NAVSEA93/NUWC)  of  the  Subma¬ 
rine  Technology  Office  (SUBTECH)  were  briefed  on  the  benefits  of  AADA.  This 
briefing  and  subsequent  working  sessions  with  the  NUWC  SUBTECH  technical 
advisor,  Mr.  Ron  Pikul,  resulted  from  discussions  with  RADM  Young  at  the  Sub¬ 
marine  Technology  Symposium  in  April  2000.  SUBTECH  documents  were  ana¬ 
lyzed  and  the  structure  of  the  decision  problem  faced  by  the  SUBTECH  manage¬ 
ment  office  was  found  to  match  the  structure  of  AADA.  Four  follow-up  meetings 
were  held,  one  included  a  detailed  discussion  on  how  AADA  would  assist  the 
decision  makers  and  described  the  cost  benefits  and  risk  reduction  value.  Even 
though  the  decision  analyses  are  applicable,  no  funding  has  been  made  available 
in  FY2001  to  transition  the  AADA  process  to  SUBTECH. 

7.3  CNO-N74-UNDERSEA  WARFARE 

CAPT  G.  Ferguson,  N74,  presented  the  status  of  the  Navy  Undersea  Warfare  chal¬ 
lenge  at  the  National  Defense  Industrial  Association  (NDIA)  Undersea  Warfare 
Symposium  in  Groton,  CT  in  September  2000.  The  decision  problem  faced  by 
N74  parallels  the  structure  of  AADA.  The  USW  example,  described  in  section 
6.3,  was  derived  in  part  as  a  result  of  CAPT  Ferguson’s  presentation.  Meetings 
were  conducted  with  N74  personnel  (including  the  Science  Advisor,  Mr.  Barry 
Raff)  and  ONR  362  in  December  2000.  Follow-up  meetings  were  held  in  January 
2001  and  the  applicability  of  AADA  to  assist  in  the  formulation  and  process  for 
the  decisions  associated  with  an  Integrated  Sponsor  Program  Plan  (ISPP)  for  Un¬ 
dersea  Warfare  (USW)  was  confirmed.  Initial  funding  of  $10K  has  been  identified 
by  the  N74  Science  Advisor.  Discussions  with  ONR  362  confirmed  that  additional 
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funding  originating  with  N74  or  a  surrogate  is  necessary  to  initiate  and  complete 
an  application  of  AADA  to  the  USW ISPP  problem. 

7.4  OUSD-AT&L 

Three  briefings  of  AADA  to  Mr.  George  Leineweber  were  held  from  September 
through  December  2000.  Mr.  Leineweber  understands  the  decision  structure  of 
AADA  and  its  applicability  to  DoD  acquisition  programs.  He  has  provided  multi¬ 
ple  potential  contacts  and  program  personnel  as  candidates  for  application  of  the 
AADA  process.  Following  up  with  these  contacts  has  to  this  date  not  produced 
a  viable  funded  interest.  Mr.  Leineweber  suggested  that  the  program  be  briefed 
to  Dr.  Paris  Genalis,  Undersea  Programs  director  for  OUSD-AT&L;  this  will  be 
scheduled  during  follow-on  efforts. 

7.5  NAVAIR 

Cost  analysts  within  the  Naval  Air  Systems  Command  (NAVAIRSYSCOM)  at¬ 
tended  an  AADA  presentation  at  the  34th  Annual  Cost  Analysis  Symposium  (34th 
ADODCAS)  in  February  2001.  Interest  in  applying  AADA  to  the  acqusition  pro¬ 
grams  for  Naval  Aircraft  was  sparked  by  the  mission  effects  based  performance 
factors  used  in  AADA.  The  decision  faced  by  these  analysts  concerns  the  number 
and  types  of  aircraft  to  acquire  constrained  by  a  budget  with  an  optimized  perfor¬ 
mance  against  a  variety  of  threats.  No  funding  has  been  obtained  from  NAVAIR 
as  yet,  however,  discussions  will  be  continued  throughout  FY01. 

7.6  BMDO 

The  Ballistic  Missile  Defense  Organization  (BMDO)  cost  analysts  are  also  inter¬ 
ested  in  AADA  as  a  result  of  the  34th  ADODCAS  briefing.  The  decision  struc¬ 
ture  faced  by  BMDO  analysts  correlates  highly  with  the  AADA  structure.  No 
progress  has  been  made  up  to  this  time  in  selecting  a  representative  problem  from 
the  BMDO  office.  Follow-up  discussions  and  collaboration  are  intended. 

7.7  OUSD-PA&E 

Mr.  Steven  Miller  of  the  Office  of  the  Undersecretary  of  Defense,  Programs, 
Analysis  and  Evaluation,  (OUSD,  PA&E)  requested  the  AADA  team  to  brief  the 
project  at  the  34th  ADODCAS.  This  briefing  was  part  of  the  Advanced  Track 
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Figure  7:  GUI  design  for  new  AADA  client  software 

training  program  at  the  symposium.  Mr.  Miller  was  introduced  to  AADA  in 
December  1999  and  has  been  kept  informed  of  its  development  on  a  continuing 
basis. 

At  the  34th  ADODCAS,  Major  Robert  (Rob)  Flowe,  USAF,  made  the  obser¬ 
vation  that  a  close  relationship  exists  between  the  AADA  process  and  the  Evo¬ 
lutionary  Acquisition  (EA)  process  currently  espoused  by  DoD  in  newly  revised 
acquisition  guidance  represented  in  DoD  Instruction  5000.  AADA  readily  fits  into 
providing  support  to  the  EA  decision  points  and  can  provide  a  quantifiable  con¬ 
nection  to  operational  effectiveness.  Follow-up  discussions  with  Major  Flowe  and 
other  DoD  acquisition  professionals  will  continue  in  FY01. 

8  Follow-On  Contract  Objectives 

After  having  predelivered  a  network-centric  AADA  software  prototype  (see  sec¬ 
tion  5)  that  was  originally  scheduled  for  completion  in  a  Phase  II  option  phase,  we 
now  intend  to  use  the  contract  replacing  the  Phase  II  option  to  turn  this  prototype 
into  a  more  mature  and  practical  form.  In  doing  so,  we  can  draw  both  on  our 
experiences  with  setting  up  and  running  AADA  example  applications  (see  section 
6)  and  on  feedback  from  potential  customers  and  other  interested  parties  to  whom 
we  demonstrated  the  AADA  prototype  software.  The  intention  is  for  the  software 
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to  become  an  effective  marketing  tool  in  its  own  right. 

This  means  that  most  short-term  development  work  will  concentrate  on  im¬ 
proving  the  AADA  front  end,  i.e.,  the  AADA  client  software,  but  also  on  user 
documentation  which  will  become  easily  accessible  online.  As  for  the  client  im¬ 
provements,  the  planned  new  interface  will  require  less  navigation  in  hierarchical 
tree  structures  and  will  display  data  in  tabular  form.  This  will  facilitate  more  con¬ 
venient  data  entry  and  will  enable  AADA  users  to  keep  track  of  ‘the  big  picture’. 
A  demonstration  screenshot  of  the  new  interface  is  shown  in  figure  7. 

The  following  specific  improvements  are  planned  at  present: 

•  make  client  sufficiently  convenient  for  effective  practical  use 

-  extensive  use  of  tables 

-  provide  default/guide  values  for  TOC,  effectiveness  and  risk  when  en¬ 
tering  effects  of  upgrades 

-  present  input  screens  as  ‘forms’  generated  from  a  few  basic  inputs 

-  allow  copying  and  pasting  of  data  between  AADA  and  commercial 
spreadsheets 

•  provide  complete  structure  editing 

-  allow  convenient  construction  of  new  examples  from  scratch 

-  allow  convenient  addition  and  deletion  of  objectives,  scenarios,  assets, 
and  upgrades 

•  make  client  more  transparent  to  new  users 

-  provide  online  help 

-  use  of  ‘tool  tips’  to  explain  some  basic  AADA  concepts 

•  enhance  AADA  functions 

-  allow  selective  ‘fixing’  or  ‘blocking’  of  investments 

-  implement  colors  of  money  -  different  budgets  for  different  classes  of 
upgrades 

-  automatic  basic  sensitivity  analysis 
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Appendices 


A  System  Design  Plan 

A.l  Introduction 

This  document  defines  a  functional-level  design  and  architecture  for  the  prototype 
Affordable  Acquisition  Decision  Aid  (AADA)  software  system.  The  primary  goal 
of  this  effort  is  a  flexible  and  extendable  decision  aid  that  incorporates  the  afford¬ 
ability  constraints,  dependency  modeling,  and  allocation  optimization  approaches 
set  forth  in  Phase  I  of  this  SBIR  effort.  Important  design  goals  are: 

•  The  ability  to  determine  near-optimal  acquisition  strategies  in  the  presence 
of  uncertainty  throughout  the  many  factors  that  define  an  acquisition  sce¬ 
nario, 

•  The  ability  to  estimate  the  overall  utility  of  a  collection  of  assets  resulting 
from  an  acquisition  strategy  in  terms  of  their  mutual  cooperation  in  meeting 
high-level  Navy  objectives, 

•  The  ability  to  recommend  acquisition  strategies  that  employ  affordability 
constraints  in  conjunction  with  the  outputs  of  the  two  previous  goals, 

•  A  database  structure  in  which  scenario-specific  and  general  parametric  de¬ 
scriptors  of  acquisition  scenarios  are  stored, 

•  A  network-centric  user  interface  via  which  users  can  access  and  control  all 
functions  of  the  system. 

The  second  objective  will  be  achieved  by  simulating,  at  a  manageable  level  of  de¬ 
tail,  the  operational  allocation  of  potential  assets  to  strategic  objectives.  In  partic¬ 
ular,  the  system  will  support  decisions  concerning  allocation  of  Science  &  Tech¬ 
nology  (S&T)  funds  to  support  future  Navy  needs.  Key  functions  of  the  system 
will  include: 

•  An  acquisition  scenario  editor  supporting  user  specification  and  modifica¬ 
tion  of  acquisition  options,  including 

-  quantification  of  expert  inputs 


20 


-  expected  Total  Ownership  Cost  (TOC)  of  assets  to  be  acquired 

-  the  nature  and  structure  of  objectives  to  be  met 

-  the  expected  performance  of  assets  in  achieving  those  objectives 

-  uncertainty  in  future  objectives  (e.g.,  uncertainty  as  to  which  future 
threats  will  materialize  and  will  need  to  be  counteracted) 

-  the  technological  uncertainty  for  S&T  investments 

•  An  automatic  optimization  capability  based  on  available  acquisition  options 
and  scenario  data  defined  by  the  user,  resulting  in  near-optimal  acquisition 
strategies,  subject  to  affordability  and  viability  constraints.  Optimality  will 
be  with  respect  to  measures  of  effectiveness  (MOEs)  which,  given  a  can¬ 
didate  acquisition  strategy,  will  be  suprema  of  user-specified  functionals 
whose  parameters  come  from  a  database  of  general  and  scenario-specific 
data.  Metrics  of  uncertainty  (e.g.,  low-order  moments  of  their  probability 
distributions)  will  be  associated  with  the  values  of  MOEs  whose  functionals 
include  uncertain  data. 

•  A  postoptimization  analysis  capability  that  will  permit 

-  viewing  of  acquisition  recommendations 

-  querying  of  reasons  behind  system  recommendations 

-  comparison  of  alternative  recommendations,  some  of  which  may  be 
defined  by  the  user 

-  determination  of  how  the  effects  of  uncertainty  about  particular  pa¬ 
rameters  in  a  scenario  are  manifested  in  the  recommendations 

A  key  property  of  the  system  will  be  network-centric  operation.  The  AADA 
system  will  be  hosted  on  remote  servers  which  take  heavy  processing  and  storage 
requirements  away  from  the  client  side.  User  interaction  with  the  AADA  sys¬ 
tem  will  take  place  through  Commercial  Off-The-Shelf  (COTS)  technologies  like 
standard  Web  browsers  and,  more  specifically,  XML  and  Java.  Thus,  the  client 
software  will  be  compatible  with  a  large  number  of  platforms  and  configurations, 
ranging  from  portable  devices  to  powerful  workstations.  Multi-user  collaboration 
will  be  supported  by  shared  access  to  a  centrally  managed  database  which  also  fa¬ 
cilitates  system  administration  and  allows  future  extensions  to  support  automated 
backup  functions  and  secure  access  control. 


21 


Main  objective 


Figure  8:  Sample  objective  hierarchy 

A.2  User  interface 

A.2.1  Acquisition  scenario  editor 

An  acquisition  scenario  will  be  subdivided  into  a  sequence  of  three  layers  of  data: 
projected  future  objectives,  presently  available  assets,  and  potential  acquisitions. 
Each  layer  of  data  definitions  must  be  completed  before  entering  subsequent  lay¬ 
ers.  Definition  of  data  in  each  layer  will  be  with  respect  to  data  in  the  preceding 
layers. 

•  Layer  1  -  Projected  future  objectives.  Definition  of  future  objectives  (such 
as  meeting  projected  future  threats)  will  precede  all  other  data  input  as  ob¬ 
jectives  will  be  central  in  defining  future  force  requirements,  which  in  turn 
will  set  the  framework  for  the  acquisition  process.  Two  principal  interlock¬ 
ing  data  structures  will  be  provided  by  the  AADA  system  for  defining  the 
nature  of  future  objectives: 

-  The  basic  objective  set.  These  will  be  the  basic  objectives  to  which  fu¬ 
ture  assets  must  be  allocated.  A  basic  objective  might  be,  for  instance, 
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acquiring  Naval  superiority  in  a  regional  theater  of  operations.  The 
basic  objective  set  will  be  the  same  throughout  a  given  acquisition 
problem,  with  the  relative  weights  of  individual  objectives  changing 
depending  on  the  scenario  under  consideration. 

-  Objective  scenarios.  Scenarios  may  be  defined  in  a  hierarchical  style 
in  the  following  way.  They  may  be  basic,  in  which  case  they  will 
be  represented  by  an  allocation  of  weights  to  the  individual  basic  ob¬ 
jectives,  or  they  may  be  composite,  in  which  case  they  will  consist 
of  a  collection  of  sub-scenarios,  each  given  a  weight  to  determine  its 
importance  with  respect  to  its  peers,  and  a  probability  of  occurrence 
(which  may  be  1)  given  the  occurrence  of  the  parent  scenario. 

Arbitrarily  complex  objectives  can  be  constructed  from  these  building  blocks 
A  sample  objective  hierarchy  is  shown  in  Figure  8. 

•  Layer  II  -  Presently  available  assets.  Next  to  the  future  objectives,  assets 
presently  available  will  be  the  most  important  reference  point  for  defining 
acquisition  requirements.  Each  available  asset  will  be  defined  in  terms  of 

1.  its  TOC,  annualized 

2.  its  probability  of  achieving  a  projected  basic  objective  as  defined  in 
Layer  I  above  (e.g.,  probability  of  kill  versus  a  projected  threat) 

3.  probability  of  its  loss  during  assignment  (e.g.,  probability  of  coun¬ 
terkill) 

4.  its  importance 

•  Layer  III  -  Potential  acquisitions.  Potential  acquisitions,  which  will  include 
upgrades  of  current  assests,  will  be  described  in  terms  of  the  following  pa¬ 
rameters 

1.  deployment  cost 

2.  degree  of  improvement  provided  by  acquisition,  compared  to  existing 
assets 

3.  change  in  TOC  to  existing  assets  after  acquisition 

4.  degree  of  technological  uncertainty  (probability  of  achieving  improve¬ 
ment  specified  above) 
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In  addition,  a  total  budget  will  be  specified  as  a  hard  constraint  for  acquisi¬ 
tions. 

Complete  layers  may  be  stored  and  retrieved  (to/from  a  central  database),  in 
order  to  facilitate  cooperative  development  of  acquisition  scenarios  and  to  allow 
experimentation,  e.g.,  by  varying  acquisition  options  assuming  a  given  definition 
of  objectives  and  a  fixed  existing  force  structure. 

A.2.2  Optimization  engine 

The  interface  to  the  optimization  engine  will  be  more  straightforward  than  the 
scenario  editor.  An  initial  screen  will  present  the  user  with  the  option  to  redefine 
default  optimization  parameters  such  as 

•  maximum  optimization  runtime  -  the  best  acquisition  strategies  found  within 
the  runtime  limit  will  be  reported 

•  number  of  recommendations  to  be  reported  back,  top-to-bottom 

•  size  of  gene  population  in  the  genetic  algorithm  (GA)  optimizer 

•  ratio  of  genetic  recombinations  to  mutations 

A  start  button  will  initiate  the  optimization  process  on  the  AADA  server. 

The  second  screen  associated  with  the  Optimization  Engine  will  provide  real¬ 
time  controls,  such  as 

•  return  to  scenario  editor,  terminating  the  optimization  process 

•  report  partial  results,  cutting  short  the  optimization  process  (generally  with¬ 
out  considering  all  feasible  acquisition  options) 

Upon  completion,  the  optimization  engine  will  transfer  control  to  the  postop¬ 
timization  analysis  tool,  starting  it  off  with  a  high-level  view  of  the  top  acquisition 
recommendations. 

A.2.3  Postoptimization  analysis  tool 

The  postoptimization  analysis  tool  will  provide  two  principal  functions,  inspec¬ 
tion  and  comparison  of  acquisition  recommendations: 
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•  Inspection  of  acquisition  recommendations.  Initially,  the  analysis  tool  will 
present  a  top-level  view  of  the  best  acquisition  recommendations,  specify¬ 
ing  the  subset  of  potential  acquisitions  selected  along  with  the  total  deploy¬ 
ment  cost.  In  addition,  for  each  recommendation,  the  asset  performance  will 
be  reported  both  as  a  whole  and  with  respect  to  the  top-level  breakdown 
into  subobjectives.  For  each  such  subobjective  that  is  further  subdivided 
into  component  subobjectives,  a  button  will  be  provided  for  displaying  per¬ 
formances  versus  the  components  and  so  on,  recursively.  For  each  basic 
subobjective,  a  button  will  be  provided  for  displaying  the  allocation  of  as¬ 
sets  to  the  corresponding  basic  objective  set.  Reported  performances  will 
typically  include  both  effectiveness  measures  and  uncertainty  metrics. 

•  Comparison  of  acquisition  recommendations.  Two  alternative  recommen¬ 
dations  may  be  compared  side  by  side,  highlighting  the  advantages  and  the 
disadvantages  of  one  allocation  versus  another.  Any  errors  in  input  that 
may  have  caused  unreasonable  recommendations  can  be  traced  to  their  ori¬ 
gin  and  amended. 

A.3  Measure  of  effectiveness  (MOE) 

The  nature  of  the  MOE  for  a  given  choice  of  acquisition  strategy,  i.e.,  its  esti¬ 
mated  level  of  performance  versus  alternative  acquisition  strategies,  will  be  cru¬ 
cial  in  defining  and  understanding  the  behavior  of  the  AADA  system  and  will  be 
important  for  predicting  the  computational  requirements  of  AADA’s  optimization 
engine.  The  AADA  MOE  will  measure  acquisition  performance  versus  objec¬ 
tives  in  terms  of  likelihood  of  achieving  objectives  and  expected  cost  in  achieving 
them.  Objectives  may  be  defined  as  hierarchical  structures  of  subobjectives  to  any 
desired  degree  of  complexity.  Each  subobjective  will  have  a  weight  defining  its 
importance  and  will  contribute  to  the  parent  objective  relative  to  its  weight  (see 
figure  8).  The  MOE  computation  will  be  structured  accordingly.  The  overall  MOE 
of  an  acquisition  strategy  will  be  just  the  degree  to  which  the  overall  objective  will 
be  met  by  the  future  assets  following  the  acquisition.  For  an  objective  composed 
of  subobjectives,  the  overall  MOE  will  be  a  weighted  combination  of  the  MOE 
with  respect  to  each  of  the  subobjectives.  For  an  objective  which  is  not  itself  com¬ 
posite,  the  acquisition  scenario  data  will  define  a  set  of  weights  for  a  fixed  set 
of  basic  objectives.  The  AADA  system  then  performs  a  GA-based  optimization 
to  determine  near-optimal  allocations  of  future  assets  to  these  basic  objectives. 
Based  on  the  predicted  performance  of  individual  assets  versus  individual  basic 
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objectives  -  and  the  weights  of  those  basic  objectives  -  the  MOE  computation  will 
determine  the  degree  to  which  the  objectives  are  met  as  whole.  It  will  be  possible 
to  associate  metrics  of  uncertainty  that  characterize  statistical  distributions  with 
both  weights  and  performance  estimates.  Thus,  we  have  a  complete  hierarchical 
MOE  computation  composed,  ultimately,  of  a  multitude  of  independent  allocation 
problems  involving  uncertain  parameters. 

The  total  number  of  allocation  problems  to  be  solved  will  be  proportional  to 
the  number  of  subobjectives  which  are  not  themselves  parents  of  composite  sub¬ 
objective  nodes.  Considering  that  each  subobjective  will  be  defined  individually 
by  an  AADA  user,  the  increase  in  computational  requirements  relative  to  that  of 
the  single  allocation  case  of  the  Phase  I  product  remains  manageable  in  the  sense 
that  there  will  be  no  combinatorial  explosion  introduced  by  the  hierarchy. 


A.4  Modular  system  decomposition  and  data  flow 

A  data  flow  diagram  is  presented  in  figure  9. 

AADA  Client  AADA  Server 


Figure  9:  Diagram  of  AADA  system  modules  and  data  flow 
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A. 4.1  Client  side  modules 

1 .  Acquisition  scenario.  This  module  will  process  user  input  to  define  acqui¬ 
sition  scenario  data  as  described  in  section  A.2.1.  It  will  also  have  access 
to  a  central  database  for  storing/retrieving  data  to  be  used  for  revision  and 
experimentation.  The  data  produced  by  the  editor,  including  control  options 
(see  section  A.2.2),  will  become  the  input  for  the  optimization  engine. 

2.  Postoptimization  analysis  tool.  This  module  will  receive  run-time  status 
reports  from  the  optimization  engine.  It  will  allow  detailed  review  both  of 
AADA  recommended  and  user-defined  allocations.  A  link  back  to  the  editor 
will  be  provided  for  the  identification  and  correction  of  errors,  as  well  as 
further  experimentation.  The  postoptimization  analysis  module  will  have 
the  ability  to  store  the  final  state  of  the  project  to  the  central  database  for 
later  revision.  The  top  recommendations  and  optimization  status  report  may 
be  saved  locally,  as  well  as  remotely.  A  hardcopy  may  also  be  produced. 

A.4.2  Server  side  modules 

1 .  Optimization  engine.  This  module  generates  near  optimal  acquisition  strate¬ 
gies  based  on  the  input  data  obtained  via  the  acquisition  scenario  editor.  The 
optimization  engine  consists  of  two  optimization  layers. 

The  upper  layer  starts  by  determining  the  maximal  and  feasible  acquisition 
strategies  -  feasible  in  the  sense  that  they  respect  budget  constraints,  maxi¬ 
mal  in  the  sense  that  no  further  individual  acquisitions  can  be  made  without 
violating  those  constraints.  The  filtering  of  viable  acquisition  strategies  in 
this  sense  is  shown  in  Figure  10. 

For  each  such  choice,  the  upper  layer  determines  the  expected  impact  on 
existing  assets,  such  as  improvements  to  their  performance  with  respect  to 
specific  Navy  objectives,  or  TOC  improvements. 

In  order  to  determine  the  quality  of  a  given  acquisition  strategy,  the  lower 
optimization  layer  simulates  allocation  of  the  resulting  assets  (taking  into 
account  any  improvements)  to  Navy  objectives.  This  is  done  by  employing 
a  GA-optimization  algorithm  using  the  MOE  described  in  Section  A.3  as 
the  genetic  viability  function. 

During  optimization,  this  module  generates  status  information  for  use  by 
the  postoptimization  analysis  tool. 
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Upper  Optimization  Layer 


Figure  10:  Upper  optimization  layer 


2.  Database  module.  This  module  will  store  and  organize  project  data  for 
shared  access.  Data  exchange  will  be  performed  directly  with  the  client 
modules. 

A.5  System  implementation  and  deployment 

The  prototype  system  will  be  developed  on  a  Pentium-based  PC  under  Redhat 
Linux.  As  discussed,  user  interaction  with  the  system  will  be  entirely  via  a  Web 
browser.  As  a  result,  the  client  component  will  be  supported  by  any  computer 
with  a  Java-capable  browser  and  network  access  to  the  server.  Parts  of  the  AADA 
server  component  will  be  developed  in  Java.  Other  parts  will  be  written  in  a 
higher-performance  language  (C++).  The  server  portion  of  the  prototype  system 
will  be  readily  portable  to  other  Unix"^ /Linux  platforms  and  will  also  be  com¬ 
patible  with  Windows  NT. 


28 


B  AADA  Descriptive  Guide  for  Analysts 


Objectives 

Assignments 

Assets/Resources 


“Fitness”  Improvements 


Ranked  sets 
(portfolios)  of 
near  optimal 
investments 

f 

AADA 


Introduction 

Class  of  problem  - 

Choice  of  portfolio  of  near  Strategy/Goals 

optimal  set  of  investments  to  /  context  \  Ranked  sets 

accomplish  a  strategy  or  set  of  '  . - (portfolios)  of 

F  bJ  Objectives  near  optimal 

£oas  ..  ,  .  ,  TT  investments 

Methodology  -  Uses  Assignments  g&p’ 

mathematical  model  that  Mm 

mimics  natural  selection  to  Assets/Resources  HP 

select  individual  investments  y/  “Fitness”  Improvements  AADA 

that  result  in  optimal  or  near  - - - — - 

optimal  fitness  scores.  Fitness  Improvement/Investment  Population 

values  are  established  based  upon  the  contextual  or  environmental  criteria.  The  core  of 
the  existing  computational  aid  is  a  genetic  algorithm  (GA).  The  GA  aims  to  breed  a 
selective  group  of  individual  investments  that  produce  "offspring"  better  than  the  parents. 

Analysis  -  Requires  decomposition  of  the  problem  into  a  particular  form.  The 
problem  set-up  is  critical  to  effective  use. 

AADA  is  an  aid  to  computation  that  may  need  to  be  modified  from  its  current 
(8/2000)  embodiment  of  the  algorithm  to  accommodate  user  unique  problems.  A  generic 
model  and/or  building  blocks  for  user  assembly  and  implementation  may  be  available  in 
the  future. 


Problem  type 


Goals  -  The  computational  objective  is  to  obtain  Pareto  optimality  for  n 


investments  in  an  n-1  dimensional 
portfolio  space.  Pareto  optimality  is  the 
joint  maximal  advantage  of  pairs  and 
because  the  GA  searches  for  local 
optimality  of  groups  the  results  will  be 
near-optimal  selection. 

Decision  characteristics  - 
AADA  in  its  current  embodiment  solves 
multiple  criteria  decision-making  (DM) 
problems  concurrently  for  multiple 


X  advantage 


objectives  and  multiple  attributes. 

Complexity  type  -  AADA  solves  dynamically  complex  problems  and  is  an 
"effects"  based  model.  This  is  as  opposed  to  Campaign  Analysis  (CA)  models  that  solve 


detail  complex  problems  based  on  attrition. 

Knowledge  acquisition  -  A  foundation  to  the  solution  of  any  analysis  problem  is 
the  collection  and  interpretation  of  information  from  subject  matter  experts.  AADA  is  as 
subject  to  the  Garbage  In  -  Garbage  Out  (GIGO)  problem  as  any  other  computational 
tool.  The  research  team  has  identified  several  valid  methods  and  sources  for  obtaining 


data  that  has  integrity  at  the  precision  needed  for  AADA  computations.  It  is 
recommended  that  users  consult  with  the  research  team  to  address  this  pivotal  area. 

Contextual  Definition 

Hierarchy  -  The  current  AADA  structure  is  rooted  in  "super  goals",  vision  or 
ideals  such  as  National  Security,  hypothesized  future  worlds  or  other  comprehensive  but 
meaningful  high  level  objectives.  Strategic  concepts  such  as  those  used  by  the  Shell  Oil 
Company  or  Value  (e.g.  USAF)  based  concepts  form  the  highest  computationally 
significant  level. 

Scenarios  -  While  AADA  uses  a  scenario  concept  it  is  not  singular  as  in  CA 
models.  Multiple  scenarios  to  the  extent  needed  to  adequately  reflect  the  intention  of  the 
high  level  objectives  can  be  defined  for  AADA  computation.  Each  scenario  must  be 
defined  in  terms  of  the  attributes  and  measures  that  are  used  in  the  achievement  of  goals. 
For  example,  if  accomplishing  the  high  level  objectives  is  based  on  achieving  and 
maintaining  likely  superiority  in  a  regional  conflict,  within  a  given  acquisition  cost,  total 
ownership  cost  and  likelihood  of  success,  then  these  factors  must  be  represented  in  some 
cost  sink  and  probability  of  success  information  at  the  lower  levels  of  the  problem.  This 
will  be  illustrated  in  more  detail  below. 

Core  analysis 

Mapping  and  consumption  -  The  GA  algorithm  requires  that  when  a  mapping, 
i.e.  a  basic  scenario,  of  resources  or  assets  (RA)  to  objectives  or  threats  (OT)  occurs  that 
for  any  given  mapping  the  resources  or  assets  are  consumed  at  the  required  level  and 
cannot  be  used  again  in  that  mapping. 

Multiple  assignment  options  -  Multiple  RA  can  be  assigned  to  individual  OT 
and  vice  versa  as  long  as  the  RA  fitness  attributes  are  not  completely  consumed. 

Objectives/threats  (OT) 

Examples  -  OT  could  be  individual  platforms  or  sites,  regional  areas  or  at 
whatever  level  of  abstraction  that  the  analyst  establishes,  as  long  as  they  are  consistent  in 
scope  among  one  another. 

Characteristics  and  attributes  -  OT  have  probability  of  occurrence,  numbers  in 
inventory,  lethality  (importance),  hardness,  economic  value  or  similar  metrics. 

Resources/Assets  (RA) 

Examples  -  RA  could  be  the  same  types  of  objects  as  the  OT,  however,  they  must 
be  logically  mappable  to  either  neutralize  or  accomplish  the  respective  OT. 

Characteristics  and  attributes  -  RA  have  value,  numbers  in  inventory, 
likelihood  of  accomplishing  the  objective  for  each  type  of  OT,  risk  in  attempting  to 
accomplish  the  objective  for  each  type  of  OT,  cost  of  ownership  or  similar  metrics. 

Improvements/investments  (I2) 

Examples  -  The  investment  or  improvement  portfolio  will  be  made  up  of 
whatever  the  program  manager  needs  to  select.  These  objects  must  serve  to  affect  the  RA 
characteristics  and  attributes  in  a  positive  (or  negative)  way. 

Characteristics  and  attributes  - 12  have  cost  and  benefits  to  the  RA 
characteristics  and  attributes  relative  to  the  OT. 
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(Near)  Optimization 

Ideal  -  The  ideal  optimization  would  be  the  portfolio  of  investments  and 
improvements  that  provides  the  best  cost/benefit  ratio  in  absolute  terms.  Because  the 
individual  selections  are  chosen  to  be  part  of  a  given  portfolio  in  a  probabilistic  fashion 
with  respect  to  their  fitness  the  AADA  solution  will  provide  "highly  fit"  individuals  but 
not  necessarily  "the  best". 

Choices  for  real  decision-makers  -  AADA  provides  candidate  portfolios  or 
groups  with  a  measure  of  fitness  to  the  decision-makers'  criteria.  It  is  easy  to  rerun 
AADA  to  establish  the  sensitivity  of  the  particular  problem  to  variations  in  input  data. 

Summary  and  conclusions 

This  document  is  a  draft  descriptive  guide  intended  to  begin  the  process  of 
knowledge  transfer  from  the  research  team  into  the  user  and  application  community.  At 
this  time  early  adapting  users  will  need  support  from  the  core  team  because  several 
documented  but  resolvable  issues  need  to  be  addressed  by  the  joint  user/research 
community.  Given  the  ease  of  use  of  the  actual  tools  and  processes,  as  the  knowledge 
base  and  user  documentation  mature  it  is  expected  that  the  AADA  tool  will  be  easily 
employed  by  analysts  with  minimal  additional  training. 

More  information: 

Please  contact  Jim  Byrnes  or  Gerald  Ostheimer  at  Prometheus  Inc.  (401)  849- 
5389,  e-mail  atjim@prometheus-inc.com.  For  further  analysis  information  please  contact  Joe 
Kranz  or  Denis  Coffey  at  A&T,  Inc.  (401)  849-5952  x31 14,  e-mail  at  jkranz@anteon.com. 

This  work  was  sponsored  by  the  Office  of  Naval  Research  (Katherine  Drew)  in  response  to  Small 
Business  Innovation  Research  proposal  number  97122043  of  1  February  1999. 

AADA™  is  a  trademark  in  the  registration  process  by  Prometheus  Inc. 
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